A service-oriented architecture (SOA) provides resources for performing particular functions as independent services that are accessible in a standardized way over a network. The services are considered standardized because distributed program components can invoke the services without any knowledge of an underlying platform that supports the services. Communication and coordination of services with one another allows the services to perform activities for entities that are tenants of the SOA.
One example of a service in an SOA is a hosting service. The hosting service may provide resizable compute capacity so that web-scale computing is easier for developers to implement. A typical hosting service provides a developer with computers on which the developer can run applications. For example, a developer can upload applications, libraries, data, and associated configuration settings to a storage repository and, using an interface, the developer can request a number of virtual machines (i.e., server instances), on which the developer can load the applications. Each virtual machine may provide a particular level of central processing unit (CPU) capacity. The service may further allow the developer to increase or decrease how much CPU and storage capacity is needed to run the applications.
When one of the developer's applications exposes a service to a client computer, a host server is determined and performs the service. In a typical hosting service, each host server in a server fleet exposes its Internet protocol (IP) address to the client computer through a domain name system (DNS) record. Using the DNS record, the client selects a host server. For example, the client might implement a round robin technique to make the selection.
Having the client select a host server presents several disadvantages. First, the client might send a request to a host server that is already processing other requests and, as a result, the host server may experience processing delays or even fail due to receiving a large volume of traffic at one time. Second, to enable the client to select the host server, a developer must manage the DNS record that identifies available host servers and populate the record with correct network addresses of the host servers. Third, the developer must monitor the host servers and update the network addresses that are stored in the DNS record when a host server fails.
To mitigate some of the potential problems outlined above, the developer might use load balancers on the client side to balance requests across the host servers. Load balancers distribute work between two or more computers or other resources to optimize resource use, throughput, or response time. However, using a load balancer on the client side presents other challenges for the developer. In particular, the developer will need to configure and run the load balancer and, in the event of a failure of the load balancer, the developer will need to replace or repair the load balancer. Accordingly, even using a load balancer requires the developer to stay involved with the configuration, maintenance, and daily operations of the load balancer.
As is evident from the above, a typical hosting service presents several disadvantages that limit efficiency and requires a developer to expend time and resources to perform monitoring and maintenance activities. Accordingly, there is a need for improved systems and methods that allow developers to concentrate on their applications so that they do not need to spend time configuring, monitoring, and maintaining client computers and/or load balancers.